Repository navigation
fix(chat): tolerate missing fields in prompt_tokens_details - #483
Conversation
OpenAI-compatible providers (e.g. DeepSeek, Gemini) send only cached_tokens or only a subset of the fields. Decoding then failed and the whole usage was silently dropped by the try? in ChatResult/ChatStreamResult. Default the missing counters to 0. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
nezhyborets
left a comment
nezhyborets
left a comment
There was a problem hiding this comment.
Thanks for the PR, please see the comment
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Done, now via This comment was drafted with Claude Code. |
There was a problem hiding this comment.
🟡 Changes recommended
Missing detail fields still cause default-configured clients to discard the entire usage object, and the regression test’s relaxed parsing setting does not cover this default behavior.
1 open issue
What changed in this PR
This PR aims to preserve chat usage data when OpenAI-compatible providers omit prompt-token detail fields, without changing the public Int property types.
Changes:
- Adds custom decoding for prompt-token details using parsing options, shared by chat and streaming responses.
- Adds a regression test for a response containing only
cached_tokens, with relaxed parsing enabled.
| File | Description |
|---|---|
| Tests/OpenAITests/ChatResultTests.swift | Tests usage decoding when audio_tokens is omitted. |
| Sources/OpenAI/Public/Models/ChatResult.swift | Adds parsing-option-aware decoding for prompt-token details. |
Review effort: Balanced
|
Also just ran the checks to see if they find anything that needs fixing |
Declaring init(from:) in the struct body suppressed the synthesized memberwise initializer; restore it as a public init, matching CompletionTokensDetails.
|
Added This comment was drafted with Claude Code. |
What
ChatResult.CompletionUsage.PromptTokensDetailsnow decodes withdecodeIfPresent, defaulting a missingaudio_tokensorcached_tokensto0. A regression test decodes a response whoseprompt_tokens_detailscontains onlycached_tokens.Why
audioTokensandcachedTokenswere decoded as requiredInts. OpenAI-compatible providers (DeepSeek, Gemini's compat endpoint) often send onlycached_tokens. Decodingprompt_tokens_detailsthen threw, and thetry?that guardsusageinChatResultandChatStreamResultswallowed the error, so the entireusage(includingprompt_tokens,completion_tokens,total_tokens) came backnilinstead of just lacking the missing detail.Affected Areas
ChatResult.CompletionUsage.PromptTokensDetailsdecoding, which is used by bothChatResultandChatStreamResult. The property types stayInt, so there is no public API change. (Making themInt?, likeCompletionTokensDetails, would be an alternative but source-breaking.)Tests: full
swift testin aswift:6.2container (223 tests) passes; the new test fails without the change (XCTUnwrapgets anilusage). No linked issue.This change was prepared with the help of an AI assistant (Claude Code), and the author is responsible for the change.
🤖 Generated with Claude Code